[WIP]Make cloud providers dynamic #15537
Draft
+57
−39
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
djyasin
changed the title
webknjaz
reviewed
Sep 19, 2024
|
Let's untangle this ball of yarn... PYTHONDONTWRITEBYTECODE=1 py.test -p no:cacheprovider --create-db --cov --cov-report=xml --junitxml=reports/junit.xml awx/main/tests/unit awx/main/tests/functional awx/conf/tests awx/sso/testsSo this is what's failing ^ And below is a traceback chain: Traceback (most recent call last):
File "/usr/lib64/python3.11/logging/config.py", line 400, in resolve
found = getattr(found, frag)
^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'awx.main' has no attribute 'utils'This one is not fully clear where it originates. Perhaps, it's pytest's collection stage. During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/logging/config.py", line 552, in configure
formatters[name] = self.configure_formatter(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/logging/config.py", line 664, in configure_formatter
result = self.configure_custom(config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/logging/config.py", line 479, in configure_custom
c = self.resolve(c)
^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/logging/config.py", line 402, in resolve
self.importer(used)
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/assertion/rewrite.py", line 184, in exec_module
exec(co, module.__dict__)
File "/awx_devel/awx/main/utils/__init__.py", line 14, in <module>
from awx.main.utils.licensing import get_licenser # noqa
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/assertion/rewrite.py", line 184, in exec_module
exec(co, module.__dict__)
File "/awx_devel/awx/main/utils/licensing.py", line 38, in <module>
from awx.main.constants import SUBSCRIPTION_USAGE_MODEL_UNIQUE_HOSTS
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/assertion/rewrite.py", line 184, in exec_module
exec(co, module.__dict__)
File "/awx_devel/awx/main/constants.py", line 8, in <module>
from awx.main.models.inventory import InventorySourceOptions
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/assertion/rewrite.py", line 184, in exec_module
exec(co, module.__dict__)
File "/awx_devel/awx/main/models/__init__.py", line 12, in <module>
from ansible_base.resource_registry.fields import AnsibleResourceField
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/assertion/rewrite.py", line 184, in exec_module
exec(co, module.__dict__)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/ansible_base/resource_registry/fields.py", line 1, in <module>
from django.contrib.contenttypes.models import ContentType
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/assertion/rewrite.py", line 184, in exec_module
exec(co, module.__dict__)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/contrib/contenttypes/models.py", line 139, in <module>
class ContentType(models.Model):
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/models/base.py", line 129, in __new__
app_config = apps.get_containing_app_config(module)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/apps/registry.py", line 260, in get_containing_app_config
self.check_apps_ready()
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/apps/registry.py", line 138, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.This is an import happening before the django things got initialized ^ The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/var/lib/awx/venv/awx/bin/py.test", line 8, in <module>
sys.exit(console_main())
^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/config/__init__.py", line 201, in console_main
code = main()
^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/config/__init__.py", line 156, in main
config = _prepareconfig(args, plugins)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/config/__init__.py", line 341, in _prepareconfig
config = pluginmanager.hook.pytest_cmdline_parse(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
raise exception.with_traceback(exception.__traceback__)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
teardown.throw(exception) # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/helpconfig.py", line 105, in pytest_cmdline_parse
config = yield
^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/config/__init__.py", line 1140, in pytest_cmdline_parse
self.parse(args)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/config/__init__.py", line 1494, in parse
self._preparse(args, addopts=addopts)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/config/__init__.py", line 1398, in _preparse
self.hook.pytest_load_initial_conftests(
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
raise exception.with_traceback(exception.__traceback__)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
teardown.throw(exception) # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/warnings.py", line 151, in pytest_load_initial_conftests
return (yield)
^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
teardown.throw(exception) # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/capture.py", line 154, in pytest_load_initial_conftests
yield
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pytest_django/plugin.py", line 361, in pytest_load_initial_conftests
_setup_django(early_config)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pytest_django/plugin.py", line 237, in _setup_django
django.setup()
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/__init__.py", line 19, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/utils/log.py", line 76, in configure_logging
logging_config_func(logging_settings)
File "/usr/lib64/python3.11/logging/config.py", line 823, in dictConfig
dictConfigClass(config).configure()
File "/usr/lib64/python3.11/logging/config.py", line 555, in configure
raise ValueError('Unable to configure '
ValueError: Unable to configure formatter 'json'And this is the last one that cascaded from the previous two ^ I see I think that before this new import |
webknjaz
reviewed
Sep 20, 2024
webknjaz
reviewed
Sep 20, 2024
webknjaz
reviewed
Sep 20, 2024
webknjaz
reviewed
Sep 24, 2024
webknjaz
reviewed
Sep 24, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
| @@ -914,23 +914,6 @@ class InventorySourceOptions(BaseModel): | |||
|
|
|||
| injectors = dict() | |||
|
|
|||
| SOURCE_CHOICES = [ | |||
| ('file', _('File, Directory or Script')), | |||
There was a problem hiding this comment.
Do we have these text explanations anywhere? Or do we need to have them in the plugins interfaces?
There was a problem hiding this comment.
I have been unable to locate the explanations in the repo, I think we may need them in the plugins interface. I will triple-check, though.
There was a problem hiding this comment.
I haven't found these in the moved-out plugins either. So yes, we should consider including this information in the plugins interface. We could temporarily put this mapping into the interfaces repo for now, or have it somewhere in serializers. I'm not sure where it's displayed.
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 27, 2024
webknjaz
reviewed
Sep 27, 2024
webknjaz
reviewed
Sep 27, 2024
webknjaz
reviewed
Oct 1, 2024
webknjaz
reviewed
Oct 1, 2024
webknjaz
reviewed
Oct 1, 2024
webknjaz
reviewed
Oct 1, 2024
webknjaz
reviewed
Oct 1, 2024
webknjaz
reviewed
Oct 1, 2024
webknjaz
reviewed
Oct 1, 2024
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
djyasin
force-pushed
the
make_cloud_providers_dynamic_28722
branch
from
3ec409f
to
bdc7d22
Compare
webknjaz
reviewed
Oct 4, 2024
webknjaz
reviewed
Oct 4, 2024
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
webknjaz
reviewed
Oct 4, 2024
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
webknjaz
reviewed
Oct 4, 2024
| @@ -192,7 +193,7 @@ def create_reference_data(source_dir, env, content): | |||
|
|
|||
|
|
|||
| @pytest.mark.django_db | |||
| @pytest.mark.parametrize('this_kind', CLOUD_PROVIDERS) | |||
| @pytest.mark.parametrize('this_kind', discover_available_cloud_provider_plugin_names()) | |||
There was a problem hiding this comment.
Looking into the test failures, it seems to me that my suspicion was likely right and this might need to be moved away from import-time invocation. And I'd try converting this to use pytest-subtests.
webknjaz
reviewed
Oct 4, 2024
webknjaz
reviewed
Oct 4, 2024
webknjaz
reviewed
Oct 4, 2024
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
webknjaz
reviewed
Oct 4, 2024
webknjaz
reviewed
Oct 4, 2024
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
|
Please retry analysis of this Pull-Request directly on SonarCloud |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY
ISSUE TYPE
COMPONENT NAME
AWX VERSION
ADDITIONAL INFORMATION